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Abstract. We focus on derangement characters of GL(n,q) which depend solely 
on the dimension of the space of fixed vectors. This family includes Thoma characters 
which become asymptotically irreducible as n —>■ oo. We find explicit decomposition of 
Thoma characters into irreducibles, construct further derangement characters and seek 



for extremes in the family of derangement characters. 



1 Introduction. Let G n = GL(n, q) be the group of invertible matrices of degree 
n with entries from the finite field ¥ g . Denote l n the unit matrix and for g G G n let 
r(g) = dim ker (g — l n ) be the number of Jordan blocks to the eigenvalue 1. We will 

O . consider Thoma characters 

O 

a 

a 
> 

Our interest in the derangement characters is motivated by the Thoma-Skudlarek 
classification of characters of the infinite group GL(oo, q) = UG n . As was conjectured 
by Thoma [5] and then proved by Skudlarek [3], any positive definite class function / : 
GL(oc, q) — >• C satisfying /(loo) = 1 can be uniquely represented as a convex combination 
of the functions o^ ■ % where a^ is the normalized version of (1) and x is a linear character. 
For the special linear group SL(oo, q) the one-dimensional factor is trivial and all positive 
definite class functions are of derangement type. From a somewhat different perspective, 
the Thoma-Skudlarek result identifies all possible pointwise limits of (rather complicated) 
characters of finite groups G n and suggests that derangement characters are the objects 
of their own right. 



(1) at\g) = q kr{g) k = 0,1,... 

and more general derangement characters which share with (1) the property that they 
depend on g only through r(g). 



By a character we shall mean a positive definite class function on a group. All char- 
acters form a convex cone, which in case of finite group has traces of complex irreducible 
representations as extreme elements. Thoma characters are extreme as functions on the 
infinite linear group, but for fixed n they can be decomposed already within the family 
of derangement characters. In Section 3 we decompose Thoma characters in ip- characters 
and then in Section 5 give explicit decomposition of the -^-characters into irreducibles. 
The ^-characters can be further decomposed within the family of derangement characters, 
thus it is natural to ask about extreme elements of the cone of derangement characters. 
One curious observation we make here is that the cone of derangement characters is sim- 
plicial for some n and not simplicial for other (the first nonsimplicial case appears for 
n — 7). Although the problem of describing all extreme derangement characters for all n 
remains open, we developed an algorithm to compute them for n < 12 (with the exception 
of n — 10) and derived a formula for stable r— characters which are extreme and span a 
\n/2\ -dimensional face of the (n + l)-dimensional cone of derangement characters. 

A straightforward generalization of Thoma characters are exponential functions z n ~ r( - 9 \ 
In section 6 we give decomposition of these functions and give a new proof to Skudlarek's 
result that only values z = q~ k yield positive definite functions for all n. In Section 8 we 
describe a unique derangement character with trivial unipotent part. All results of this 
paper hold for arbitrary finite field. 

2 Characters ajr . Let XJ^ be the set of n by k matrices over ¥ q . The group G n acts 
on Xj? 1 by left multiplication. For k — 1, 2, . . . let o^. be the character of the permutation 



y(") 



. We also define <7q to be the unit character of G n . 



representation in 

Proposition 1 The characters a}! 1 coincide with the Thoma characters defined by (1). 

Proof. The value of a^ (g) is equal to the number of matrices x G X^ fixed by g. 
Clearly, x is fixed if and only if each column of x belongs to the space V g C F" of g- 
invariant vectors. Since the dimension of V g is r(g), there are q r ^ possible choices for 
each of k columns of x . □ 

Define a function of g G G n to be derangement if it depends on g only via r(g). 
Obviously, any derangement function is a class function. We denote D n the space of 
derangement functions; this is a complex vector space of dimension n + 1. 

The object of our interest is the cone D+ of positive definite derangement functions 
/ t^ 0. Each / G D+ is called a character and dim/ = /(l n ) is called the dimension of / 



(even when / is not a trace of a matrix representation of G n ). Because matrices g and g^ 1 
have the same eigenvectors we have r(g) = r(g~ r ), therefore any derangement character 
accepts only real values which are always in the range |/(<?)| < dim/. 

Proposition 2 The characters {cr^ '■ < k < n} form a linear basis of D n . 

Proof. Choose arbitrary gj G G n with r(gj) = j , < j < n. The matrix of degree n + 1 
with entries 

is nondegenerate, because it is the Vandermonde matrix in variables 1, q, . . . , q n . It follows 
that the n + 1 characters are linearly independent and form a basis. □ 

We shall view G n as the generic term of the increasing series of groups G\ C G2 C . . . 
with the natural embedding which sends g G G n to g © 1 G G n+1 . Since the embedding 
adds fixed vectors we have 

\ l ) °k |g„_i - q °k ■ 

Taken together with Proposition 2 the restriction rule implies that the class of derange- 
ment functions is closed under restriction to smaller groups. 

3 Characters i^k- Thoma characters are not irreducible. The first obvious step to 
decompose them is to split X^ 1 into orbits. Let V fc be the space of n by k matrices 
over ¥ g with rank k. For k — 1, . . . , n define f/4 to be the character of the permutation 
representation in C [Y^ : ] and for k — let ip^ = 1. 

Proposition 3 The characters ip^ 1 are derangement. They are given by the formula 

(3) ^\g) = (q r - l)(q r - q )...( q *- g*" 1 ), r = r{g) 

(which is for r(g) < k). 

Proof. For V g being as in Proposition 1, a matrix y G Y" fc ^ s h xe d by g if its columns are 
in V g and linearly independent. Counting the choices compatible with the independence 
condition, we have q r — 1 possible choices for the first column, then q r — q choices for the 
second, etc. □ 

The following branching rule is analogous to the restriction formula (2). 



Proposition 4 For 1 < k < n we have 

(4) tfW, = M n_1) + ((T*- 1 - ^- 1 )^-i 

ft/ie /irst term zs wid /or k = n). 

Proof. Fix o G G n -i with r(o) = o. The embedding G n -\ C G n sends g to o © 1 with 
r(gffil)=r + l. We shall count the number of matrices x E Y^ n fixed by g © 1. Let x 
be such a matrix, and x be this matrix with the last row deleted, then of course gx = x 
and the rank of x must be either k or k — 1. 

In the first case there are ip^ 1 (o) choices for x which could be arbitrarily combined 
with any of q k choices for the last row of x. This yields the first term in (4). 

In the second case x can be seen as a /c-tuple of column vectors which span a (k — 1)- 
dimensional subspace of V g (dimV^ = r). Making further distinction between the cases 
when the first column of x belongs to the space spanned by the rest k — 1 columns or not 
we compute the number of choices for x as 

( q r-l)...( q r- q ^) q —Zl^ 

q-1 

which is a multiple of i^k-i (fl 1 ) D Y Proposition 3. Since the rows of x also span a (k — 1)- 
dimensional space, there are always q k — q k ~ x ways to extend itoa matrix of rank k by 
appending the last row. This results in the second term in (4). □ 

Recall that the g-binomial coefficient is defined as 

fk\ {q k - l)(q k ~ 1 - 1) ■ ■ ■ (q k ~ j+1 - 1) 



\jj q (qi-l)(q^-l)...(q-l) 

and is equal to the number of j-dimensional subspaces in ¥ k (which is for k < j). 



Proposition 5 The characters a^ and ip^' are related by the formulas 

(5) «?' = £ ( k ) i-r 

j=0 \JJq 

(6) ^b-ii'-v'^W 1 

7=0 \J J a 



Proof. We decompose X k into G n -orbits. Let x G X k be a matrix of rank j with 
columns seen as elements of F™. Select j linearly independent columns, label them 
v\, . . . , v j and label Vj+i, . . . , v k the rest columns. Consider the space of linear relations 
a in k indeterminates over ¥ q such that a(v i, . . . , Vk) = and let «i, . . . , a n -j be a basis 
of this space. Since the natural action of G n in F" is j-transitive (for j < n) the orbit 
{gx : g G G n } coincides with the set of matrices whose columns satisfy the a^s and the 
first j columns are independent. It is easily seen that the orbit of x is isomorhic to Y k . 
Using the isomorphism X k kF"®fJ, the linear span of the orbit becomes F™ ® A where 
A C Fg is the null space for relations ctj. 

Any choice of independent columns results in the same A, therefore the correspondence 
between the orbits and subspaces of F^ is bijective. It follows that X k splits into ( .) 
orbits isomorphic to Y- , j = 0, . . . ,n, which implies (5) (the term j = corresponds 
to the singleton orbit of the zero matrix in X k ). The second formula (6) follows from 
the first by virtue of an inversion formula for the g-binomial coefficients (analogous to a 
better known inversion formula for the binomial coefficients). □ 

It follows that the characters {^fc '■ < k < n} also form a linear basis of D n and 
that both sets of characters generate the same integral lattice. 

The equivalence of branching formulas (2) and (4) can be also derived from the ex- 
pansions (5) and (6). In fact, the relation between the two sets of characters is the 
specialization of the g-binomial formula 

(i+0(i+e?)...(i+e^) = E( fc ) q®? 

j=0 \J J q 



for £ = —q 



■-fe+i 



4 On irreducible characters of G n . We will need some well-known facts about the 
irreducible characters of G n , referring the reader to [7], [6] for a fuller account. 

As usual, we identify Young diagram A = (Ai, . . . , A m ) with its geometric image {(i,j) '■ 
1 < i < m,l < j < Aj}, and write | A| = Ai + . . . + X m for the number of boxes. We denote 
Y the collection of all Young diagrams, including the empty diagram with |0| = 0. 

Given integers 1 < k < n and two characters f\ and ji of the groups Gk and G n _fe, 
respectively, their parabolic product j\ o f 2 is the character of G n induced from the 
parabolic subgroup 
(7) P = | ( 9 * *) : 9l g G k , g 2 e G n _ k \ 



by the function fi(gi) ■ ./^(fi^) (in this context the groups are embedded in G n as suggested 
by the definition of P). The dimension of the parabolic product is 

(8) dim /i o / 2 = (J dim j\ ■ dim / 2 

as it follows from the Frobenius formula for induced characters and the observation that 
the left coset classes for P can be labeled by A;-dimensional subspaces V C F™. 

A character is called cuspidal if it is not a part of any parabolic product. We denote Cj 
the finite set of cuspidal characters of Gj and C = Ud>iCd- The unit character of G\ = F* 
plays a distinguished role and will be denoted e; it is one of q — 1 elements of C±. Given a 
family <p : C — > Y with finitely many nonvoid diagrams 0(c) 7^ 0, its degree is defined as 

IMI = ££rf-k(c)|. 

rf>l cGC d 

A fundamental fact says that the irreducible characters of G n are in one-to-one corre- 
spondence with the families of Young diagrams of degree n. We denote (tp) the character 
corresponding to such a family ip. 

The character of G n corresponding to the family with a single nonvoid diagram 0(e) = 
A is called unipotent and will be denoted (A) e . For unipotent characters the dimension 
can be computed by the g-hook formula: 

(9) dim (A) = (g- 1 )---^"- 1 ) n(A) , A | 

where /i(6) denotes the hook length Aj+A^— «— j + 1 at box b = (i,j) and n(X) = J2(i — l)Aj. 
The involved function of g is also called the Kostka-Foulkes polynomial i^A,(i™)(<?)- 

Remark. To avoid confusion between the diagram and its transpose keep in mind 
that we adopt the parametrization which relates the one-row diagram to the unit character 
(n) e , while the one-column diagram corresponds to Steinberg character (l n ) e of dimension 
-^(i"),(i n )(?) = 1 ■ I n man Y sources the convention is reverse (e.g. [6]). 

5 Decomposition of -ip^ into blocks of irreducibles. Let reg fe be the character 
of the regular representation of Gk- The next proposition says that, in a sense, characters 
ip k interpolate between the unit and the regular character of G n . 

Proposition 6 We have ip^ = (n) e , V4 n) = re gn > an d for 1 < k <n 

(10) Vi n) = (n - k) e o reg fe . 

6 



Proof. Since G n acts transitively on Y^ character ip£' is induced by the unit character 

4' 



from the subgroup B of block matrices ( 9 * ^ ) ,<?i £ G n _fc, which fix ( ( n ~ k ) xk j g Y fc 



The parabolic product structure is recognized when we view the induction as the two-step 
procedure: at first inducing from B to the parabolic group (7) - which yields the character 
(n — k) e (gi) ■ reg k (g2) of P - and then using this character to further induce from P to 

G n . □ 

Introduce the character pj = YJ \M\=j dim(</?) • (0) of Gj and for A G Y, |A| < n set 

¥>(e)=0 

(11) [A] n = (A) e op n _| A | 

Decomposition of [A] n into irreducibles follows directly from the definition: this involves 
all family of Young diagrams with 0(e) = A. 

Characters [A] n are disjoint in the sense that none of the irreducible characters of 
G n enters decompositions of two different [A] n 's. We shall see that [A]„'s are convenient 
'blocks' for representing derangement characters, they are themselve irreducible if A has 
n boxes (in which case [A] n = (A) e ). 

Given /i e Y and integer m let H+(/x) be the set of Young diagrams which can be 
derived from jj, by appending a horizontal strip. That is to say A G H+ (/x) if the skew 
diagram A \ fi has m boxes with no two in the same column. Reciprocally, let H~ (A) be 
the set of diagrams which can be derived from A by deleting a horizontal strip. Clearly, 

(12) AGH+M ^ ^GH m (A). 

Note that the largest horizontal strip has Ai boxes, thus H~ (A) is empty if the first row 
of A is shorter than m. 

Next result gives explicit decomposition into blocks. 

Theorem 7 Each character f/4 , < k < n, is an integral linear combination of the 
characters [\} n : 

4 n) = E4 n) [AW 

|A|<n 

The multiplicity is zero if \X\ > n or \\ < n — k, otherwise 

(13) 4" ) (A)=f k _ \ £ dim(^) e 

\ n l A IA M eH- (A) 



Proof. The regular character reg fc splits into irreducibles as 

reg fc = J2 dim(p) • (ip) 

||<p||=fc 
J2 J2 dim ((/ i )e°M)-(^)e°M 



\\ V \\ = k-\vi\ 



J2 dim(/i) e - (/i) e o p k _ M 

\li\<k vi/^i/ 9 

as it follows from (8) and (11). 

For 1/ G Y and integer m the parabolic product decomposes as 

(14) (m) e o(u) e = £ (A) e . 

This follows by virtue of Pieri's rule which is the same for unipotent characters of G n as 
for characters of the symmetric group. From this, Proposition 6 and (11) we obtain 



,(») 



*l>k = re Sfc ° ( n ~ k )t 



Y. i. J dim(/i) e J2 Pfe-H°( A )e = 

H 1,1 dim (/^)e H [A]„. 



\n\<k WW/ q AeH+_ fc (^) 

The coefficient at [A] n is calculated by swapping the sums, applying (12) and observing 
that n — \\\ = k — \p\ implies 

W\) q = [n - \X\) q 

D 

In one most important case the multiplicity formula (13) simplifies. Given A let v = 
A \ (Ai) be the diagram derived from A by deleting the first row. Note that deleting the 
maximum horizontal strip of A also yields v. Suppose % = X± — (n — k) > 0, then removing 
a horizontal strip with n — k boxes from A is equivalent to appending a horizontal strip 
with % boxes to v. On the other hand, appending % boxes to v results in some diagram 
p € H~_ fc (A) provided that p\ < X\. It follows that (13) is equivalent to 



» 



4 n; (A) 



{n-x) ? dlm(/i)e - 



Corollary 8 IfX 2 <n-k<X 1 th 



en 



»a\ (k\ (k j 



crw= : r . J dimz, 



v 7/ \ « , 

\J / q \ / q 



where v — X \ (Ai), i = Ai — (n — fc) and j = n — |A|. If n > 2k then for any A mt/i at 
most n foxes 



» 



E (,) ( ■ "J E dimz/-[n-A; + 

+j<* \J/n\ J a M=k-i-i 



(15) C ; = E ■ • E dimi/.[n -* + »,!/]. 

i+j<k \J/q\ / q \i/\=k—i—j 

i,j>0 

Proof. Suppose A2 < n — A; < Ai then A2 + i = X 2 + Ai — (n — k) < \± and therefore 
A*i < ^1 f° r an y A* *= H^"(z/). By Pieri's rule and (8) we get 

^ dim (/i) e = dim (z/) e o (i) e = . dim (z/) e 

M£H+(A) V * A 

because \u\ + i — |A| — A x + 2 = (n — j) — (i + n — k) + i = k — j. 

If n > 2/c the inequality Ai > n — fc implies X 2 < n — X\ < n — (n — k) = k < n — k. 
Now (15) follows because any diagram entering the decomposition of ^>Jj. is of the form 
A = (n — k + i, v) with \v\ < k — i.O 

Remark. The coefficient (13) is a multiple of the skew Kostka polynomial 
^.\\(n-jfc),(i fc -j)(9)> as introduced in [1]. Under conditions of Corollary 8 the skew diagram 
X\(n—k) splits in two parts with no common boxes in the same row or column; factoring of 
the polynomial also follows from the interpretation as the generating function of tableaux 

(see [1]). 

A positivity property of Kostka- Foulkes polynomials implies that (13) are polynomials 
with positive integral coefficients. 

Corollary 9 (i) Character [A] n with the first row X\ = n — k is present only in the 
decomposition of ipk , ■ ■ ■ , ^^ ■ 

(ii) The empty diagram enters only the regular character, so that c k , (0) = 5kn (Kro- 
necker delta). 

(iii) We have for the regular character 

9 



(iv) For one-row diagrams we have c k (n — k) = 1, and more generally for 'hook dia- 
grams' A = {n — k + i, l fc- ' - -?) with i,j>0;i+j<k< n: 



( n ) i k\ (k — i\ tk-i-j\ 



Proof. Straightforward from (13). For (iv) apply (15). □ 

Example. We tabulate coefficients of the decomposition into [A]„'s for n = 4. 





W 


w> 


w 


W 


W 


(4) 


1 


1 


1 


1 


1 


(3,1) 





1 


1 + q 


1 + q + q 2 


q + q 2 + q 3 


(2 2 ) 








1 


q + q 2 


q 2 + q 4 


(2, I 2 ) 








Q 


q + q 2 + q 3 


q 3 + q 4 + q 5 


(I 4 ) 











q 3 


q" 


(3) 





1 


1 + q 


1 + q + q 2 


1 + q + q 2 + q 3 


(2,1) 








l + q 


l + 2q + 2q 2 + q 3 


q + 1q 2 + 1q 3 + 2q A + q 5 


(I 3 ) 











q + q 2 + q 3 


q 3 + q 4 + q 5 + q 6 


(2) 








1 


1 + q + q 2 


l + q + 2q 2 + q* + q A 


(I 2 ) 











1 + q + q 2 


q + q 2 + 2q 3 + q 4 + q 5 


(1) 











1 


1 + q + q 2 + q 3 

















1 



Any derangement function / has a unique representation as a linear combination 
of the ipf. n s and this implies a decomposition of / into [A] n 's. In particular, Theorem 

(n) 

7 combined with Proposition 5 enables representing Thoma characters o~ k as integral 
linear combination of the [A] n 's. 

6 Generalized Thoma characters. Given z £ C consider the derangement function 
fz(g) = z n ~ r ^ 9 '. This definition is consistent for different n because r(g © 1) = r(g) + 1, 
thus f z is defined on the infinite group G^ = UG n . Obviously, for z = 0, f z is the 
normalized regular character equal to Si^^g , while for z = q~ k it is the normalized Thoma 
character 

m ■- ° k - n -k{n-r{g)) 

k '~ J- (n) ~ q 

dim a k 
Skudlarek proved that for z ^ the only positive definite functions among f z are 
Thoma characters (see [3], Behauptung 3). His proof exploited an embedding of the 

10 



additive group of infinite matrices into G^. We show next that this result follows rather 
easily from the decomposition of f z into irreducible characters for n — 1, 2, . . .. 

Proposition 10 For zee 

i% / j — 1 i \ 

as) /,=e K-''n4^ H n) - 

i=o V i= o y V 

Froo/. By (5) 

j = V/ g 

for fc = 0, ...,n, which transforms into (16) with z = q~ k . But this implies that (16) 
holds everywhere because for each r(g) both parts of the formula are polynomials in z.n 

Corollary 11 The function f z : G^ — > C is positive definite if and only if z = or 
z = q~ k , k = 0, 1, 

Proof. If f z is positive definite then for each n the coefficients in the decomposition into 
[A] n 's are nonnegative. Character [0] n enters only ip^ with the coefficient being a positive 
multiple of (1 — z)(q~ 1 — z) . . . (q~ n+1 — z), which in turn is positive for all n provided that 
either z is from the conjectured list or z < 0. Hence we only need to exclude negative 
values. 

Steinberg character [l n ] n enters only ipn-i an d i'n^- By (16) and Corollary 9 (iv) the 
coefficient at [l n ] n is 

, , n—l „—i „ , , n—2 



q — z (n-i\ q - z 



For z < positivity of the coefficient amounts to the inequality 

-1 

z > 



q n _ q n-l _ I 

which has the right-hand side vanishing as n — > oo (q > 1); hence f z cannot be positive 
definite on all G„'s if z < 0.D 



' n 



7 The cone D+. All characters of G n form a cone whose extreme rays correspond 
to irreducible characters. This cone is simplicial, so that any character has a unique 
representation as a positive linear combination of the irreducibles. Since derangement 
characters are always reducible (besides (n) e ) it is natural to ask which of them are 'the 
least reducible'. 

11 



A derangement character / G Z)+ is said to be extreme if /', / — /' E Z?+ implies 
/ = p/' for some p > 0; in this case {pf : p > 0} is called the extreme ray. (Sometimes we 
make no distinction between a character and its positive multiples.) Extreme characters 
are of primary interest because each / G D+ is a positive linear combination of the 
extremes. 

Let L n be the space of class functions representable as linear combinations of the [A] n 's 
and L+ be the cone of characters in L n . 

Although [A] n 's are not derangement (see next proposition), they offer a useful coor- 
dinatization of the space of derangement functions. Indeed, because L n is generated by 
disjoint irreducible characters each / G L n has a unique representation as 

/= E /WW» 

|A|<n 

and / G L\ if and only if the coefficients /(A) are nonnegative. We will call the set of 
diagrams supp / := {A : /(A) > 0} the support of / G L+. 

Because D n C L n we have Z)+ = D„ n L+ and we can use this fact to distinguish 
the characters from other derangement functions. The cone D„ is polyhedral and has a 
compact base (a polytope). Since Z)+ contains linearly independent characters ^ , < 
k < n, the number of extreme rays must be at least n + 1, and if it is exactly n + 1 
the cone is simplicial and each character has a unique representation as a positive linear 
combination of extremes. 

By Corollary 9 (i) we see that 

supp il)£' = {A : Ai > n — k}, 

and this implies an important observation that the supports are strictly increasing with 
k. An immediate consequence is 

Proposition 12 None of the characters [X] n is derangement, besides the unit character 
[n] n - 

Proof. Given / G D n , let n — j be the length of the shortest first row of all A's entering / 
with some nonzero coefficient. Since the supports are increasing, j is the maximum index 
of the nonzero a^'s entering f = Y, a kWk ■ -^ u ^ then all [A] n with Ai = n — j enter / with 
the same Oj. Now, if / = [A]„ then Ai is the shortest first row of all diagrams. But for 
< Ai < n there are other diagrams with the same first row which would enter in the 
derangement case. That [0] n G" D n will follow from Theorem 17. □ 

12 



Let Ck be the cone obtained via intersecting the space spanned by V'o > • • • ■> vu with 
D+. The cones Ck are increasing with k, and from the increasing of supports follows that 
each cone Ck is a (k + 1)— dimensional face of D+, whence the following claim. 

Lemma 13 f E Ck is extreme in D+ if and only if f is an extreme character in Ck- 

Given a finite set {f) : j G J} C L n and % G J we say that A is an eigendiagram of fi 
if A G supp fi but A £ U je j\{j} supp /,. 

Lemma 14 C^ zs simplicial if and only if there is a list of characters fo, . . . , fk G Ck 
such that each fi has an eigendiagram (in which case this is the complete list of extremes). 

Proof. Suppose each f\ has an eigendiagram, then the linear mapping which assigns 
fo, ■ ■ ■ , fk to the basis vectors of R fe+1 is an order isomorhism sending Ck to the positive 
orthant. It follows that there are no other extreme characters. 

Conversely, suppose Ck is simplicial and let fo, . . . , fk be a complete list of extreme 
elements such that fo has no eigendiagram. In this case supp fo C U^o supp fj thus 
selecting a^ > sufficiently large we obtain some / = J2j^o a jfj — fo £ Cfe- We get then 
for fo + f G Ck one decomposition J2i^o a ifi without f Q , while decomposing / we get 
another decomposition fo + f = fo + J2 hfi (with 6j > 0) which does involve fo- Since for 
simplicial cone the decomposition into extremes must be unique we have a contradiction. 
□ 

There is a simple method to verify if a character is extreme. 

Lemma 15 Let f - - - , f n be a basis of D n with f G D+. The character f is extreme 
if and only if the column vectors 



{fj{\) : A^supp/o} .7 = 1,..., 



/; 



are linearly independent. Hence each extremal character must have at least n zero coeffi- 
cients. 

Proof. If linear independence does not hold there is a linear combination / = Y^=i a jfj 
such that /(A) = for all A G supp fo- Selecting e > we obtain a character / = fo — ef G 
D+ with supp /' G supp / . For p sufficiently large pfo — /' G D+. Thus fo is not extreme 
since the characters fo and /' are not colinear. 



13 



If the linear independence does hold there is no noncolinear character / with supp / C 
supp /o, because there is no linear combination as above. □ 

These considerations motivate introducing yet another basis Tq , . . . , r^f 1 E D n which 
we define recursively, as the output of the following elimination algorithm. 



The elimination algorithm. Set r (n) := ip^K At each stage k ^- l,...,n we 
<k 
setting at first t^q :— ipk an< ^ f° r 3 — ^-i- • • i^ — \ 



have characters r) , j < k, at hand and determine sequentially characters r^o, • • • ,Tkk by 



_ (n) 

T kj ■— T k,j-1 — a k,j-l T j-l 

where the coefficient a,k,j-i takes the maximum possible value compatible with the con- 
dition that the difference be a character (i.e. in L+). Finally, define r^ := Tkk- 

Remark. Explicitly, the coefficients are 

dij = minrfc J (A)/rj_ 1 (A) 

(a/0 = oo). Complemented by a^j = 5ij,i < j, they determine the transition matrix from 
the basis {4 n) } to {4 n) }. 

To apply the algorithm one needs to determine the minima like mm P\(q) for certain 
polynomials in q. However, there is a computer evidence that this problem is trivial: the 
polynomials involved have positive coefficients and there is always a polynomial which 
has minimal coefficients at all powers of q. 

Proposition 16 If the cone Ck is simplicial then t[ j , . . . , r^ is the complete list of 
extreme characters of Ck- 

Proof. The statement is trivial for k = 0. If Ck is simplicial then the same applies to 
Cfc_i, so suppose by induction that Tq , . . . , r^_ x are extreme in C^-i (thus, by obvious 
extension of Lemma 13, are extreme also in C&). Note that Ck is in the linear span of 
Yk i r o j • • • i T k ■ Consider the step resulting in Tki- In geometric terms, the elimination 
means that we determine the intersection point of the ray connecting r and ip k with 
the face Q C Ck not containing r . Obviously, Q is a simplicial cone of lower dimension, 
the characters rj , . . . , r^" are extreme in Q and the linear span of Tki, t\ , . . . , T k n 
contains Q. This is the same situation as with Ck but now the dimension is reduced, thus 
the induction step can be completed. □ 
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Combining Lemma 14 and the last proposition, we see that D+ is simplicial provided 
each 7)j: has an eigendiagram, otherwise not. Next example illustrates the approach. 

Example. The decomposition of characters r^ (n — 4) is 

-(4) (4) (4) (4) (4) 

r r l r 2 r 3 '4 

(4) 1 

(3,1) 1 

(2 2 ) 1 

(2,1 2 ) q q 

(l 4 ) q 3 

(3) 1 1 

(2,1) l + q l + q q + q 2 

(l 3 ) q + q 2 + q 3 q 3 

(2) 1 1 l + q + q 2 

(l 2 ) l + q + q 2 q + q 2 + q 3 

(1) 1 l + q + q 2 

1 

It is seen that (4), (3, 1), (2 2 ), (l 2 ), are the eigendiagrams. Therefore Df is simplicial, 
and Tfc , < k < 4, is the complete list of extreme derangement characters. The linear 
relations between the bases are the following: 



■S 4) = 


T (4) 










■i 4) = 


r (4) 




+r x (4) 






f> = 


r (4) 




+ (9 + lK (4) 


+r 2 (4) 




•i 4) = 


r (4) 




+ (g 2 + g + l)r 1 (4) 


+(g 2 + g)r 2 (4) 


+r 3 (4) 


■i 4) = 


r (4) 


+(g E 


' + q 2 + q + l)rj 4) 


+(g 4 + g 2 )r 2 (4) 


+g 3 r 3 (4) +r| 4 ) 



Results of similar computations for n < 22 are as follows: 

(i) For n = 1, 2, 3,4, 5, 6, 8, 9, 11, 12 the cone D+ is simplicial and {r^ n , < k < n} is 
the complete list of extreme characters. 

(ii) For n = 7, 10, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22 the cone is not simplicial. All char- 
acters t^ are extreme but this list is not complete. 

(hi) For n = 7, 10, 13, 14, 15, 16, 17, 18, 19, 20, the number of characters r^ with no 
eigendiagram equals one, for n = 21 this number is two and for n = 22 it is three. 
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In all above cases if r^ ,k<n, has an eigendiagram it is of almost rectangular shape 
((n — k) a , b) where n = a(n — k) + b (i.e. the shape differs from rectangular only in the 
last row). 



Example. The minimum n such that D+ is not simplicial is n = 7. There are 9 

(7) 

extreme derangement characters: r^ , < k < 7, and one additional character 



T. 



(7) .. 



(7) , (7) 



-(7) 



where 



a i 



(1 + q + q 2 ) 



and a-i 



1 + q)(l + g 2 ) 2 (l + q + q 2 + q 3 + q 4 
q 2 + q 4 + q 5 + q 6 + q 7 + q 8 + q 10 



The table shows the pattern of positive coefficients at unipotent [A] n (with diagrams of 
full degree n) and [0] n 



sign 

7) 
6,1) 

5,2) 

5,1 2 ) 
4,3) 
4,2,1) 
4, I 3 ) 

3M) 
3,2 2 ) 

3,2,1 2 ) 

3,1 4 ) 
2 3 ,1) 
2 2 ,1 3 ) 
2,1 5 ) 

I 7 ) 




T, 



(7) (7) (7) (7) (7) (7) (7) 




+ 



















T 























+ 
+ 
















T: 








+ 
+ 
+ 













n 









+ 
+ 
+ 
+ 









TV 









+ 
+ 



+ 
+ 
+ 
+ 
+ 
+ 






T, 



6 





+ 



+ 
+ 




+ 
+ 
+ 
+ 
+ 
+ 





J7) 
T 7 

















+ 

(7) 



ri 7) 




+ 



+ 
+ 
+ 



+ 
+ 



+ 
+ 



The eigendiagrams are (7), (6, 1), (5, 2), (4, 3), (3 2 , 1), (l 7 ), 0, but r 5 has no eigendiagram. 
Each row in the rest of the coefficients matrix is a positive linear combination of the rows 
of the above block. The completeness of the list of extreme characters was shown with the 
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help of Lemma 15. The base of the D^f is a polytope which is combinatorially equivalent 

(7*1 (7) (7) (7) 

to a 5-fold pyramid build upon the 'square' {r 4 , r 5 ,r* , r 6 }. 

Our computations strongly suggest the following 

Unipotent conjecture: each row of the coefficients matrix corresponding to a dia- 
gram with less than n boxes is a positive linear combination of the rows corresponding to 
unipotent characters and to [0] n (for n < 22 it was sufficient to take unipotent characters 
with almost rectangular shape). An equivalent property is that the cone dual to D+ is 
spanned by positive combinations of the rows of the unipotent block. Equivalently, the 
projection r i— ► J2\x\=n T W[Mn is an isomorphism of ordered spaces. 

8 The character with no unipotent part. Introduce the character 

(17) e } = E (^m 1 ) dim(A) e -[A] n . 

|A|<n-l V l A l / q 

In all cases covered by the computational results of previous section, this character coin- 
cides with t$™\ Although we failed to prove that the coincidence is not incidental we will 
show that f^ n - ) is indeed derangement and give it characterization. 
Denote n q = 1 — q n and n q \ = l g 2 g • • • n q . 

Theorem 17 Character f^ is extreme. It can be characterized as the unique (up to 
a scalar multiple) derangement function orthogonal to all unipotent characters [A] n . In 
terms of the basis derangement characters: 

(n) _ ,/.(n) _ V ^~ 1 ^ ! „*,/,W 



18) f (») =V ,W_ S v2_p: g * v , 



k ' 
fc=o N- 



-k 



Note that the sum in (18) is alternating, since k q < for q > 1. The proof of this 
result is based on one nontrivial identity. 

Lemma 18 For each diagram A with n boxes 

n—l I -I \ | 

(19) cW(A)-E i! 4T k ^ n) ( A ) =0 

fc=0 K i- 
Proof. Schur functions satisfy 

(20) SA (Ui,6,...) = E^(6,6,---) 
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where the summation is over the set of diagrams H~(A) = {/x : A \ (Ai) C \x C A} which 
can be derived from A by deleting a horizontal strip (the term with \x — A is also included 
in the right-hand side). Similar formula with m variables amounts to the branching rule 
for characters of GL(m, C). Specializing the Schur function for £_,■ = g- 7 we have 

2 x dim(A) e 
n q \ 

and by homogeneity s^(q, q 2 , . . .) = g M s M (l, g, g 2 , . . .) (see [4], p. 375). 

In view of Theorem 7 the left-hand side of (20) is c^(X)/n q \, while the right-hand 
side is 

t E <£>(A)£ 

*=0*=H-_ fc (A) N- 

which implied readily (19). □ 

Remark. Formula (19) is a hidden version of the Kostka-Foulkes polynomials identity 
found in [2], with a minor correction. Our proof is borrowed from [2] (where the formula 
needs correction by taking A in place of transpose A'). 

Example. For hook diagrams X — (n — m, l m ) the identity amounts to 

(n-A (-+i) ^ (n-iy qk ( k\ (?) 

V m J q ^o k i ] - \ m ) q 

Simplifying this becomes 

E? J ^ = i 

i=0 ^9- 

which can be proved straightforwardly by induction on N. 

Proof of Theorem 11. For A with less than n boxes set j = n — |A| and observe the 
recurrence 

(21) ct\\) = (*) clf(X) 

(which is for k < j). Plugging this into the left-hand side of (19) and applying Lemma 
18 for n' = n — j along with the q— binomial identity f n ) = ( n Z 1 ) + q^\~ ) and 
Cn-j = dim (A) e we obtain 
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which is equivalent to (18) by (17) and Theorem 7. 

It is obvious from the definition and (18) that f^ e D+. For each k — 1, . . . , n — 1 the 
character ip^' includes at least one unipotent [A] n which is not in fl^supp tpj . Hence 
the linear rank of the 'unipotent' matrix block U with entries 

U kiX = d£\\) \\\=n, 0<k<n 

cannot exceed n. The uniqueness claim now follows because there are no two noncolinear 
combinations of the basis derangement characters with zero unipotent part. We see that 
the rank of U is n, hence in accord with Lemma 15 fj^ is extreme. □ 

As a by-product we obtain: 

Corollary 19 For j = 0, 1, . . . ,n, the linear rank of the matrix blockU(n—j) = {c k "(A) : 
| A | = n — j,0 < k < n} is n — j . 

Proof. We have proved this for the unipotent block; and for other blocks this follows from 
(21) by induction on n. □ 

It is not at all obvious from the explicit formula 

ti n \g) = (-l)V^ V *m ~(n- l) q \r q \ , g E G n , r = r(g) 
that this function is positive definite. 

Remark. It would be interesting to learn if there is some intrinsic relation between 
r^(g) and regular characters. The complement in reg n is yet another derangement 
character 

reg„ - f£»> = E ? n - |A| (,"r V) dim We W„, 

A|>1 \l I / (j 

but formal replacing [A]„ by [A] n _i in (17) yields reg n _ 1 . 

(n) 

9 Stable characters. In Section 7 we introduced characters r^ implicitly, by a 
recursive procedure. For k < n/2 there is an explicit formula 



4 n) = J2 ( . ) S dim (/ i ) e ' ( n - /C ' A*]n 
.7=0 W a \u\=i 



(22) 

i=0 \ J / q \n\=j 

and the relation of these characters with i\r£ s is the same g— binomial as the relation 
between ip— and a— characters (5): 



^-sC). 



(23) ^r j = >: ; r. 
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as it follows easily from the simple case (15) of the formula for coefficients. 

To prove that these characters indeed appear as the output of the algorithm, we can 
just start by defining them by one of the two formulas. Then we observe that t^ has 
eigendiagram (n — k, k) (which other characters of this set do not have), therefore they 
span C\n/2\, they are extreme by Lemma 14 and this cone is simplicial. By Proposition 
16, the elimination algorithm gives the full list of extremes in the simplicial case, thus 
these extreme characters coincide with (22) (possibly up to a positive factor). 

From Lemma 13 follows that 

Theorem 20 The characters r^ are extreme for k < n/2. 

We wish to stress that (23) is only valid for the indicated range, and inverting the 
formula for k < n/2 would not produce positive definite functions at all. 
Inverting (23) and applying (4) we get yet another branching rule 

(24) T^\ Gn _ x = q k rt l) + 2^ V " l)^ 1 + ^"V" 1 - !)(?* - 1)^ 

(with obvious adjustments for extreme values of indices). 

Characters (22) are stable in the sense that, as n grows, the diagrams entering the 
decomposition of such a character keep changing only in the number of boxes in the first 
row. Asymptotic considerations, which lie outside the scope of this paper, show that the 
normalized characters ipf. n , r^ approach Thoma characters as n —* oo. 
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